The Social Nature of Perception and Action

Humans engage in a wide range of social activities. Previous research has focused on the role of higher cognitive functions, such as mentalizing (the ability to infer others’ mental states) and language processing, in social exchange. This article reviews recent studies on action perception and joint action suggesting that basic perception–action links are crucial for many social interactions. Mapping perceived actions to one’s own action repertoire enables direct understanding of others’ actions and supports action identification. Joint action relies on shared action representations and involves modeling of others’ performance in relation to one’s own. Taking the social nature of perception and action seriously not only contributes to the understanding of dedicated social processes but has the potential to create a new perspective on the individual mind and brain. KEYWORDS—perception and action; action perception; joint action; social cognition; social neuroscience Cognitive scientists have long believed that perception and action are two servants of the mind, residing in opposite wings of the mental mansion. According to this view, perception delivers messages from the outside world to keep the mind informed, and action executes what the mind commands. Recent research in the cognitive sciences and neurosciences suggests that the mind is actually more like a kibbutz than a mansion: Perception, action, and cognition seem to form a collective community. Perception and action are intimately linked, and cognition is firmly grounded in both of them. This new perspective has important implications for understanding the functional and brain mechanisms that support people’s ability to interact with others. Individuals do not just infer intentions, emotions, and attitudes from others’ behavior (Fiske, 1992); rather, researchers have postulated a more immediate way of social understanding and social interaction, based on the close link between perception and action. For example, when one observes another individual performing a particular action, this activates the representations in one’s own action system that one uses to perform the observed action. Taking the social functions of perception and action seriously might help to better understand disorders of social functions, including autism and certain symptoms of schizophrenia such as delusions of control. In this article, we discuss recent findings from two research domains that shed new light on the social nature of perception and action. Research on action perception demonstrates that individuals rely on their bodies and the action system moving their bodies to understand others’ actions and to identify their own actions. Research on joint action has revealed how individuals share representations, predict each other’s actions, and learn to jointly plan ahead. BODILY AND MOTOR CONTRIBUTIONS TO ACTION PERCEPTION Understanding When people watch sports games like basketball or soccer, they often find their limbs twitching as though they are taking part in the game. This indicates that observing actions can directly activate the motor system. The common-coding hypothesis provides a functional principle that can explain such phenomena. According to this hypothesis, perceiving and performing an action results in the activation of the same representations—i.e., ‘‘common codes.’’ Evidence for common coding has been found at the level of single neurons, the so-called mirror neurons. The same neurons in the premotor cortex of a macaque monkey fire both when the monkey grasps an object and when it observes an experimenter grasping the same object. The implication of this striking finding is that brain areas that were thought to be purely motor areas also support action perception. This creates a new perspective on how individuals make sense of others’ actions. Rather than understanding observed actions by mapping them onto abstract concepts, people ‘‘relive’’ them by mapping them Address correspondence to Günther Knoblich, Psychology Department, Rutgers University, Smith Hall, 101 Warren Street, Newark, NJ 07102; e-mail: [email protected]. CURRENT DIRECTIONS IN PSYCHOLOGICAL SCIENCE Volume 15—Number 3 99 Copyright r 2006 Association for Psychological Science onto their own action knowledge. This leads to an immediate recognition of the goals underlying observed actions. Numerous brain-imaging studies have demonstrated that the mirror system supports action understanding. Identification But how do observed actions get mapped onto action representations? According to the common-coding principle, the more similar an observed action is to the way the observer would perform the action, the higher the activation of action representations. This was recently demonstrated in a study by CalvoMerino and colleagues (Calvo-Merino, Glaser, Grezes, Passingham, & Haggard, 2005), in which ballet dancers and capoeira dancers watched videos of ballet dancing and capoeira dancing. Activation of the mirror system was stronger when the dancers watched the type of dancing they were experts in. The mirror system should become even more strongly activated when persons perceive recordings of actions that they themselves have previously performed, such as when seeing videos of themselves dancing or hearing recordings of their own piano playing. The higher level of activation during the perception of self-produced actions might allow individuals to distinguish their own actions from those of others (Repp & Knoblich, 2004). Several studies have shown that people are indeed able to distinguish between their own and others’ actions even when there are no cues with respect to the actor’s identity besides dynamic information about body movements or the effects they result in. For instance, when people see point-light displays of themselves and their best friend dancing, jumping, or boxing, they can identify themselves better than they can identify their friend (Loula, Prasad, Harber, & Shiffrar, 2005). If visual familiarity were the main factor, the opposite result should be observed, because one sees one’s friend’s movements much more often than one’s own from a third-person perspective. In a similar vein, people can identify their own handwriting from a single moving dot (Knoblich & Flach, 2003). One is also able to identify the results of one’s earlier performed actions in the auditory domain. It was found that nonmusicians were able to identify their own clapping (Flach, Knoblich, & Prinz, 2004). Expert piano players could distinguish a recording of a piece they had played for the first time from recordings of the same piece performed by other pianists (Repp & Knoblich, 2004). Whereas experts use subtle timing deviations from the score, known as expressive timing, to identify their playing, nonmusicians rely on tempo and salient rhythmic idiosyncrasies to identify their clapping. Thus there is converging evidence that the similarity between observed actions and the way one would perform them oneself leads to a higher activation of common codes. This, in turn, allows people to identify their own previous actions. Simulation In addition to action understanding and action identification, the mirror system seems to support the prediction of future action outcomes (action simulation). In particular, matching observed actions to one’s own action repertoire allows one to exploit mechanisms in the motor system that are normally used to predict the outcomes of one’s own actions. This solution is more parsimonious than predicting the results of others’ actions based on a separate perceptual-anticipation mechanism. In support of simulation, it was found that people observing someone throwing a dart could quite accurately predict where the dart would land. Importantly, the predictions of the landing position were most accurate when participants observed videos of themselves throwing the dart, although recording session and recognition session were at least 1 week apart. This makes it very unlikely that the higher accuracy for self was due to memories for the outcome of particular throws (Knoblich & Flach, 2003). The higher degree of similarity between a perceived action and the way one would perform it oneself led to the higher prediction accuracy for oneself than for others. There is also reason to believe that proprioceptive signals (sensing position of body parts) and tactile signals from one’s own body contribute to action simulation. A recent study showed that lacking these signals impairs action understanding (Bosbach, Cole, Prinz, & Knoblich, 2005). In this study, two de-afferented individuals were tested. De-afferentation refers to the loss of body sense due to a degeneration of all nerve fibers that normally transmit sensory information to the brain. The two individuals observed videos of an actor lifting a box. Prior to lifting the box, the actor had sometimes been told the correct weight of the box and had sometimes been deceived about its weight. Both individuals had difficulties telling whether the actor lifting the box had the right or wrong expectation about its weight. In contrast, healthy participants had no problems making these judgments. They could tell from the actor’s body posture and body movements whether or not the actor had been deceived, suggesting that action simulation in healthy individuals is supported by peripheral bodily signals. The lack of peripheral bodily signals in the de-afferented patients resulted in faulty simulations. ENGAGING IN JOINT ACTION WITH OTHERS Acknowledging the close links between perception and action also has implications for theorizing about joint action—social interactions wherein two or more individuals coordinate their actions in space and time to bring about a change in the environment. Some examples are doing the dishes together, rowing a canoe together, or playing a piano duet. Joint action involves sharing action representations and coordinating one’s actions with those of others to achieve common goals (Clark, 1996). Shared Representations While previous research has focused on the role of language and theory of mind for successful social interaction, more immediate interpersonal links may exist in the form of a common coding 100 Volume 15—Number 3 The Social Nature of Perception and Action